Singular ridge regression with homoscedastic residuals: generalization error with estimated parameters

摘要

This paper characterizes the conditional distribution properties of thefinite sample ridge regression estimator and uses that result to evaluate totalregression and generalization errors that incorporate the inaccuraciescommitted at the time of parameter estimation. The paper provides explicitformulas for those errors. Unlike other classical references in this setup, ourresults take place in a fully singular setup that does not assume the existenceof a solution for the non-regularized regression problem. In exchange, weinvoke a conditional homoscedasticity hypothesis on the regularized regressionresiduals that is crucial in our developments.